Outsmarting Go dependencies in testing code

Reading time: 9 minutes

Writing good tests is tricky when the system has a lot of moving parts. When using Go’s testing infrastructure, tests that involve multiple modules can cause dependency cycles which are not allowed by the compiler. In this post we will go over a technique we devised to break these dependency cycles.

Background

The CockroachDB Go code base is split up into various packages; some of the major ones are:

• storage: interfaces with the local stores
• kv: key-value store
• sql: SQL layer (on top of the key-value store)
• server: high level code for setting up a CockroachDB node exposing a PostgreSQL interface on a network port. A node is, among other things, both a kv and a sql server.

We will focus on just the sql and server packages. The server package depends on the sql package – as it should, since the server code sets up the SQL server part of the CockroachDB node.

Most sql tests involve setting up a test server, running some SQL statements and potentially peeking at or poking some internal implementation detail. To start a test server, we want to be able to leverage the code in server which sets everything up for us. But tests in the sql package cannot depend on the server package because that creates a circular dependency.

This problem is not specific to CockroachDB – we suspect many large go codebases could run into this problem as tests tends to use shortcuts that cross logical boundaries. After all, all’s fair in love, war, and testing code.

The Initial Solution

Our first solution was to use Go’s facility for black box testing (testing only through a package’s public interface). Go allows tests in a package like sql to be declared as being part of a sql_test package. This is a separate package as far as dependencies are concerned so it breaks the dependency cycle, allowing us to import server. The downside is that we don’t have access to the internals of sql from this package! So we were forced to export various internals for the sole purpose of accessing them from tests, or split off parts of the sql_test code and put them in sql test code.*

This got to be more and more annoying as time went on. When we started work on a new distsql package for what will become our distributed-SQL implementation, we again were forced to expose a lot of package internals for tests. It was time to investigate a better solution.

Toward a Better Solution

What we really wanted was to write tests in the sql package from where we can directly access the sql internals. The only way to call out to server code for instantiating test servers would be indirectly, through a shim layer – a module that does not depend on either sql or server but which can be used to indirectly interface between them:

We worked on a simple proof-of-concept which illustrates the idea. The server and sql packages represent the real packages as described so far. The testingshim defines an interface for the server functionality that we want to access from sql tests, but it doesn’t actually depend on either server or sql. Methods that need to use (or return) types defined in sql can do so indirectly, using interface{}:

package testingshim

// TestServerInterface defines test server functionality that tests need.
type TestServerInterface interface {
  SQLSrv() interface{}
  // Other needed stuff goes here.
}

// TestServerFactory encompasses the actual implementation of the shim
// service.
type TestServerFactory interface {
  // New instantiates a test server instance.
  New() TestServerInterface
}

This layer also holds a key piece of global state: serviceImpl can be set to an external implementor of the interface defined here (via InitTestServerFactory):

var serviceImpl TestServerFactory

// InitTestServerFactory should be called once to provide the implementation
// of the service. It will be called from a xx_test package that can import the
// server package.
func InitTestServerFactory(impl TestServerFactory) {
  serviceImpl = impl
}

func NewTestServer() TestServerInterface {
  return serviceImpl.New()
}

The idea would be that a type in server implements the TestServerFactory interface, and something that has access to both server and  testingshim calls InitTestServerFactory, allowing sql tests to call functions like NewTestServer. “Something” was where we got stuck for a while, until..

The Hack

The final piece of the puzzle also revolves around the black box testing facility that allows for a sql_test package, but used in a more ingenious way. The go test documentation states:

Test files that declare a package with the suffix “_test” will be compiled as a separate package, and then linked and run with the main test binary.

So if we had sql_test code that used server, the server code would be in there somewhere; Go just won’t allow us to access it from tests declared as part of sql. The “aha” moment was when someone pointed out TestMain(). TestMain is an optional function that can be used for doing extra setup before testing; a single TestMain can live in either the sql or sql_test package. By putting it in sql we are able to run initialization code which has access to server before running sql tests!

• Note: A possible alternative to TestMain would be to use an init() function in a sql_test file.

This is again illustrated in our proof-of-concept: in the sql_test package, TestMain has access to both the server code and the testingshim. It can initialize the TestSrvInstance global with a type implemented by server:

func TestMain(m *testing.M) {
  ..
  testingshim.InitTestServerFactory(server.TestServerFactory)
  ..
}

And that allows sql tests to use testingshim.NewTestServer():

package sql
..
func TestFoo(t *testing.T) {
   testingshim.NewTestServer().SQLSrv().(*SQLServer).Woof()
}

The dependency graph is:

The full fledged change was of course more involved, but it follows this simple recipe. This one-time effort of creating the dependency-free testingshim package was worth the ease of writing tests going forward, especially as we can easily make use of the same framework in other packages.

Go coders out there – if you hit the same problem and find this trick useful, let us know in the comments below!

*_Updated on June 17, 2016_